Melody-to-Chord using paired model and multi-task learning language modeling

In music composition, melody is relatively easy to generate: a piece composed even by an amateur can be pleasing at times. However, chords composition typically requires some musical background to have them be acoustically in harmony. In this report, we designed a system to generate chords matching a given melody using several models and draw similarities between natural language and music. We mainly focused on two models: a proposed paired model and a multi-task language model, each with a few extensions. Both models were able to generate chords that match to melody in harmony, and we found that proposed paired model performance was able to generate more pleasing music. 1 2 1 Music as a Language Music and natural language are very similar. They are both in the form of sound (musical notes and speech) which can be transcribed into symbols (musical sheet and articles). Therefore, acoustic model and language model are extensively used on both of them respectively. Moreover, one can view music theory as the grammar in natural language and preprocess their hierarchical structure. The pitches, notes, bars, musical phrases, and chorus in music are the counterpart of phonemes, morphemes, words, phrases, and articles in natural language. Given so many similarities, many researchers deem music as a universal languageCohen [2008], Besson and Schön [2001]. In our project, we want to generate chord progression given melody, which is a sequence-to-sequence problem just like the language translation in NLP. There are still some difference between natural language and music. For example, music is represented in time units in music, which allows us to stretch our data to twice the length or compress it into half the length for data augmentation and take average of predictions for more stable results. Also, music theory can be formulated using mathematics and thereby have multiple representations. Based on these advantages that only music possess, we propose paired model and language modeling using multi-task learning and merging units methods for melody-to-chord in this report. Link to generated music https://goo.gl/Mzflpl Github: https://github.com/hsuwt/cs224n-project